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ABSTRACT 



This report discusses the many issues that need to be 
considered as states and districts begin to design and implement alternate 
assessments, which are used for students with severe cognitive delays or 
multiple disabilities or students who require such significant accommodations 
that the validity of the test would be compromised. The report provides 
examples from the two states, Maryland and Kentucky, that currently have such 
programs. The report begins by describing the differences between assessment, 
a process of collecting information, and accountability, a system activity 
designed to assure those inside and outside the educational system that 
schools are moving in desired directions. Thirteen issues to be considered in 
designing an alternate assessment program are then described and discussed, 
including: (1) eligibility for alternate assessments; (2) who makes 

eligibility decisions; (3) how to maintain a unified educational system; (4) 
how to avoid overuse of alternate assessments; (5) how to assess in reliable 
and valid ways; (6) whether data should be aggregated or reported separately; 
(7) how results should be communicated; (8) linkage between what is taught, 
what is tested, and giving students an opportunity to learn what is tested; 

(9) the costs and benefits; (10) how data will be collected: (11) types of 

data that should be collected; (12) confidentiality; and (13) how to ensure 
that school personnel are trained to administer, score, interpret, and use 
the results of alternate tests. (Author/CR) 
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Executive Summary 



Accountability systems that include ^ students must have a way to measure 
the performance of students who cannot be assessed through typical testing 
procedures. Alternate assessments are designed to meet this need, which is 
estimated to apply to approximately 1 to 1-1/2% of the student population. 
There are many issues that need to be considered as states and districts 
begin to design and implement alternate assessments. Thirteen issues 
described and discussed in this report are: 



Issue 1: 
Issue 2: 
Issue 3: 
Issue 4: 
Issue 5: 
Issue 6: 
Issue 7: 
Issue 8: 


Eligibility-Who Takes Alternate Assessments? 

Who Makes Eligibility Decisions? 

How Do We Maintain a Unified Educational System? 

How Do We Avoid Overuse of Alternate Assessments? 

How Do We Assess in Reliable and Valid Ways? 

Should Data be Aggregated or Reported Separately? 

How Should Results be Communicated? 

Linkage: Do We Test What We Teach, and Do Students 
Have an Opportunity to Learn What We Test? 


Issue 9: 


What are the Cost Benefits? 



Issue 10: How Will Data be Collected? 

Issue 1 1 : What Kind(s) of Data are to be Collected? 

Issue 12: How Do We Maintain Confidentiality? 

Issue 13: How Do We Ensure that School Personnel are Trained to 

Administer, Score, Interpret and Use the Results of Alternate 
Tests? 
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Alternate Assessments in Accountability 

National, state, and local educational personnel increasingly are engaged in 
efforts to document the extent to which education is working for students. 
State and national legislatures have passed laws, rules or guidelines 
requiring that states set educational standards and assess the extent to which 
students are making progress toward those standards. In an increasing 
number of states students must pass tests in order to receive a high school 
diploma (Bond, Braskamp, & Roeber, 1996; Thurlow, Ysseldyke, & 
Anderson, 1995). The 1990s clearly is a decade of accountability in 
education, and at least part of the information used to demonstrate 
accountability comes from assessments. 

It is important at the outset of this paper to distinguish between assessment 
and accountability. Assessment is a process of collecting information, one 
form of which may be test information, for the purpose of making decisions 
about students. Accountability is a system activity designed to assure those 
inside and outside the educational system that schools are moving in desired 
directions (Center for Policy Options, 1994). Information obtained from 
assessments typically is used to demonstrate accountability. Accountability 
involves reaching consensus on clear goals for an organization or person, 
reliably assessing achievement toward those goals, and attaching 
consequences to the success or failure to achieve goals. 

A major challenge in education is to demonstrate accountability for all 
students. National and state education legislation (e.g.. Goals 2000, 
Improving America’s Schools Act, and the pending versions of the 
Individuals with Disabilities Education Act) includes language specifying 
that accountability applies to all students. Educational goals and standards 
are for aU students. States and school districts are to report on the 
performance and progress of all of their students. “All” includes students 
with disabilities and students with limited English proficiency. About 85% 
of students with disabilities have relatively mild or moderate disabilities and 
can take state and national large scale assessments, either with or without 
accommodations (like large print, testing in a separate setting, or extended 



NCEO 

O 



1 



Alternate 
assessments 
are used 
when 
students do 
not “fit” within 
the regular 
assessment 
program, or 
when the 
tests typically 
used do not 
“fit” a 
segment of 
the school 
population. 



time). Yet, there is a group of students with disabilities for whom current 
tests are inappropriate, and who, therefore, are excluded from district, state 
and national assessments. These typically are students with severe 
cognitive delays or multiple disabilities or students who require such 
significant accommodations that the validity of the test would be 
compromised. In this paper, we address issues and considerations in 
assessing this group of students. These students can be assessed through a 
practice that has become known as “alternate assessment.” Alternate 
assessments are used when students do not “fit” within the regular 
assessment program, or when the tests typically used do not “fit” a segment 
of the school population. 



What is Alternate Assessment? 

In Table 1, we show the three kinds of tests used with students with 
disabilities: general state tests, general state tests with accommodations, and 
alternate assessments. We also provide estimates of percentages of students 
expected to participate in each kind of test. An alternate assessment is a 
substitute way of gathering information on the performance and progress of 
students who do not participate in the typical state assessments used with 
the majority of students who attend schools. Typical state assessments 
involve use of criterion-reterenced assessments, standardized norm- 
referenced multiple choice tests (like the Iowa Tests of Basic Skills), 
performance tests, and sometimes portfolios. Alternate assessments can be 
the same kinds of assessments (e.g., performance measures or portfolios), 
but they probably differ in format, content, or level from the assessments 
that are used with the majority of students. We have chosen to use the term 
alternate assessment rather than alternative assessment. The term alternative 
assessment has multiple meanings, and anything that is an alternative to a 
standardized multiple choice test has become known as an “alternative.” 
So, throughout this paper we use the term alternate assessment. 
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Table 1. Three Kinds of Assessments 
Disabilities 


for Students with 




Percent of General 
Population 


Percent of Students 
with Disabilities 


General Assessments 


80-95% 


50-75% 


General Assessments 
with Accommodations 


3-5% 


10-35% 


Alternate Assessments 


1-2% 


10-20% 



Examples of Alternate Assessment Practices 

Only two states currently have alternate assessment programs: Maryland and 
Kentucky. Personnel in Texas are developing a blueprint for an alternate 
assessment, and are expected to complete it in early 1997. Many other 
states are exploring a variety of options to begin development of such a 
program. 

Maryland 

Maryland has several kinds of tests within its assessment programs. The 
Maryland Functional Testing Program includes four basic minimum 
competency tests: three multiple choice tests in reading, mathematics and 
citizenship, and a modified-holistically scored direct writing assessment of 
both narrative and explanatory writing skills. All of the tests are untimed. 
The Maryland Functional Testing Program (MFT) is a graduation 
requirement program that now is required to be completed by the end of 
eighth grade. A high school test is being developed in the same content 
areas to replace the MFT, but to reflect higher standards. Students will be 
required to pass this test in order to graduate. 

The Maryland School Performance Assessment Program (MSPAP), 
currently administered at the elementary and middle school levels, measures 
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higher order thinking processes and the application of knowledge and skills 
to real world situations. It is a single test covering mathematics, reading, 
writing, science, language usage, and social studies. 



Maryland’s performance assessment for populations with severe disabilities 
is called the Independence Mastery Assessment Program (IMAP). IMAP is 
currently being field tested in approximately one-third of the school systems 
in Maryland. It reflects an alternative set of educational outcomes, more 
life-skills oriented, that were identified and developed by the state under the 
direction of the IMAP Advisory Committee (see Table 2). Students with 
severe disabilities participate in either MSPAP or IMAP, depending on 
which outcomes they are pursuing. In the pilot districts, students with 
disabilities participate either in (a) MSPAP with no accommodations, (b) 
MSPAP with accommodations, or (c) IMAP. 



Table 2. Examples of Maryland’s Content Domain Outcomes 
for Students in Independence Mastery Assessment 
Program 

Content Domain: Indicators: 



Personal Management; Students 
will demonstrate their ability in the 
following areas: personal needs, 
appropriate health and safety practices, 
managing household routines, and 
participating in transition planning with 
adult service providers 

Community: Students will 
demonstrate their ability to access 
community resources and get about 
safely in the environment 

Career A^ocational: Students will 
demonstrate their ability to participate 
in transitioning to employment and in 
various employment opportunities 

Recreation/Leisure: Students will 
demonstrate their ability to participate 
in recreational and leisure activities 



• Eating and feeding self 

• Dressing appropriately for 
activities, season, and weather 



• Shopping or browsing for 
variety of items 

• Demonstrating safe pedestrian 
skills 

• Arriving at work appropriately 
dressed and on time 

• Completing assigned duties 
with appropriate productivity 
and quality 

• Engaging in hobbies 

• Participating in clubs or 
organizations 
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Kentucky 

Until the current year, four types of assessment tasks comprised the 
Kentucky Assessment System: assessment tasks involving portfolios, 
assessment tasks involving performance events, assessment tasks involving 
open-ended questions, and assessment tasks involving machine-scorable 
questions. Use of performance event scores for accountability was dropped 
during the 1997 school year because of questions about the reliability of 
these scores. Kentucky also monitors school progress in terms of non- 
cognitive indicators such as school attendance rates, dropout and retention 
rates, reduction in physical and mental health barriers to learning, and the 
proportion of students who make a successful transition to work, 
postsecondary education, or the military. 

Kentucky has the Alternate Portfolio Assessment for use with students who 
have severe cognitive disabilities. The following are key concepts of the 
Alternate Portfolio: 

• An Alternate Portfolio Advisory Committee, charged with 
the task of identifying the Academic Expectations to be 
assessed within the Alternate Portfolio, first looked at the 
critical functions of each of the Academic Expectations, and 
determined the extent to which each could be demonstrated 
by students eligible to participate in the Alternate Portfolio 
Process. Twenty-eight academic expectations were 
identified as critical (see Table 3 for examples of some of 
these expectations and indicators of them). 

• Scores of students participating in the Alternate Portfolio are 
weighted equally with those of students participating in the 
regular assessment for the school’s accountability purposes. 

• The student’s portfolio must include seven to ten entries 
related to the state’s academic expectations, and included 
among the entries must be: 

• a student activity schedule or routine; 

• a resume of job experiences; and 

• a sample of the student’s present mode(s) of 
communication. 
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Table 3. Examples of Kentucky’s 
Students in the Alternate 


Academic Expectations for 
Portfolio System 


Academic Expectation: 


Indicator(s): 


Accessing Information: Students 
use research tools to locate sources of 
information and ideas relevant to a 
specific need or problem. 


• Requests assistance 


Reading: Students construct 
meaning from a variety of printed 
materials for a variety of purposes 
through reading. 


• Reads environmental, 
pictorial print 


Quantifying: Students organize 
information by quantifying real, 
whole, rational, and/or complex 
numbers. 


• Counts 

• Uses one-to-one 
correspondence 


Writing: Students communicate 
ideas and information to a variety of 
audiences for a variety of purposes 
through writing. 


• Constructs printed, 
pictorial messages 

• Uses personal signature 


Constancy: Students understand the 
tendency of nature to remain constant 
or move toward a steady state in a 
closed system. 


• Predicts next event 



Issues in Alternate Assessment 

Many groups and individuals engaged in state and national assessments 
have raised issues about alternate assessments. We have derived the list of 
issues described below from multiple forums with diverse sets of school 
personnel; focus groups with school, community, legislators and business 
people; interviews; meetings with working groups; feedback at workshops 
and conferences; and our own experience. 



6 



NCEO 



Issue 1: Eligibility-Who Takes Alternate Assessments? 

Decisions about who participates in alternate assessments have significant 
ramifications. Some contend that alternate assessments are for students 
who cannot participate in typical state tests. Others contend that alternate 
assessments are for students who are working toward meeting a set of 
separate standards. Still others contend that alternate assessments are 
intended for students who are not working toward a “real” high school 
diploma. And, still others contend that alternate assessments are only for 
students who are working toward separate standards and who are not 
working toward a typical high school diploma. Therefore, the 
establishment of criteria for inclusion in an alternate assessment has 
implications for diploma awarding, graduation, and even for definitions of 
the purpose of education. 

The complexity of this issue is enhanced by the fact that standards are 
specified at various levels. When states specify broad standards, like 
“students who complete school will solve math problems with sufficient 
accuracy to be successful in their next environment,” there are multiple 
levels of expected performance, and multiple ways in which the standard 
can be assessed. When states specify narrow standards, like “All students 
will complete school with working knowledge of mathematics three years 
beyond ninth grade algebra,” there is one level of expected performance, 
and limited ways in which to assess the extent to which it has been met. 
Resolution of this issue is fundamental. School personnel must decide 
whether there will be more than one set of standards, decide how broad 
those standards will be, and then decide the extent to which students will 
work toward separate standards. How this decision relates to the type of 
diploma earned must be determined as well. 

The way in which the issue of who takes the alternate assessment is 
resolved will ultimately affect many of the other issues stated below. For 
example, if assessment personnel take the position that alternate 
assessments are for students who cannot take typical state assessments, 
how will potential overuse of the alternate assessment be limited? If, on the 
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other hand, alternate assessments are limited to students who are not 
working toward the same standards as other students, then how do we 
produce an alternate assessment that provides valid and reliable scores that 
can be used in an accountability system? 

Issue 2: Who Makes Eligibility Decisions? 

Generally, it is thought that individuals who know a student well should 
decide in which kinds of assessments the student will participate. In many 
states and districts, this decision is made by lEP teams, while, in others, the 
decision is made by teachers or building principals. Should lEP teams 
decide who participates in what kinds of assessments? If so, how much 
training or re-training will be needed to enable them to make appropriate 
decisions? Should parents decide whether their children will take tests and 
the kinds of tests they will take? 

Issue 3: How Do We Maintain a Unified Educational 

System? 

If school personnel either hold students with disabilities responsible for 
meeting separate standards, or assess them using a separate set of tests, they 
run the risk of creating or enhancing separate (often set-aside) rather than 
unified educational structures. When separate groups (even categorical 
groups) are provided educational opportunities directed toward meeting 
separate goals or standards, we end up with special education for students 
with disabilities. Title I education, bilingual education, technical education, 
education for students at risk, and “general” or “regular” education. Where 
does the fractionation of the educational system end? In an era in which 
there is a tremendous push for unified educational systems, such separatism 
typically is discouraged. 

Issue 4: How Do We Avoid Overuse of Alternate 

Assessments? 

Decisions must be made about who participates in typical assessments and 
who participates in an alternate assessment. One danger in having available 
an alternate assessment is that it may encourage school personnel to have 
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large numbers of students participate in the alternate assessment, thereby 
lowering rates of participation in typical assessments. This danger is 
especially relevant if there are less stringent consequences for the schools 
when students take the alternate assessment. It is expected that few students 
(between one-half of one percent and two percent of the school age 
population) will need alternate assessments. Most professionals agree that 
the decision about participation should not be based on expectations of poor 
performance on the general education assessment. In Kentucky and 
Maryland, students are considered eligible for alternate assessments when 
they are working on a different curriculum from most students, one that 
may not lead to a diploma. And, of course, the breadth of the standards 
toward which students are working will often help to define the kind of 
assessment system in which they participate (see Issue 1). 

Issue 5: How Do We Assess in Reliable and Valid 

Ways? 

Clearly, it is very difficult to develop technically adequate measures of skill 
development for some kinds of students in some domains. For example, it 
is difficult to get reliable and valid indices of academic and functional 
literacy for students with severe cognitive or communication impairments, 
students with multiple disabilities, or students who refuse to respond to 
tests, tasks, or interviews. It also may be difficult (and this is true for both 
typical and alternate assessment) to get technically adequate indices of 
student development in the domains of citizenship and contribution, 
responsibility and independence, and social-emotional functioning. 

Issue 6: Should Data be Aggregated or Reported 

Separately? 

Assessors regularly argue about the extent to which it is wise to aggregate 
data across multiple assessment systems (e.g., typical state assessments and 
alternate assessments), including multiple alternate assessment systems. In 
the recent past, there were many educators who believed firmly that students 
with disabilities should not be part of accountability systems. Today, most 
agree with the contention that accountability systems should be inclusive. 
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The debate now seems more focused on how to report than on who to 
assess. And, some contend that data should be reported in the aggregate 
rather than separately. 

Because nearly all students who participate in alternate assessments will do 
so because they are not working toward general education goals or 
standards, some might assume that all assessments of these students will be 
individual appraisals of the extent to which they are achieving lEP 
objectives. The primary difficulty with this approach is that lEP goals are 
not equivalent across students, and attainment of lEP goals cannot easily be 
aggregated for accountability purposes. There are common domains of 
goals or a common core of learning that all students who are unable to 
participate in the general education assessment still need to be working 
toward. The state must work to link the alternate assessment to this 
common core. 

Issue 7: How Should Results be Communicated? 

When states implement both typical tests and alternate tests as parts of their 
accountability systems, clear communication of the results is a major 
challenge. The challenge is intensified when some students take typical 
tests with accommodations. Assessment personnel must be able to 
communicate specifically who took what kinds of tests under what kinds of 
conditions, and give clear information about the level of performance of the 
students. 

Issue 8: Linkage: Do We Test What We Teach, and Do 

Students Have an Opportunity to Learn What We Test? 

Curriculum alignment is an important issue for all kinds of assessments 
used for educational accountability. It is critical that educators test what 
they teach. Given the diversity of curricula in which students are enrolled, a 
major challenge is one of making certain that what is tested is actually 
taught. Students with severe disabilities are often enrolled in curricula for 
which the goals are attainment of life skills, such as self-care, mobility and 
development of positive social interactions. It is important that alternate 
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assessments include these areas of curricular content. Defining these areas 
of the curriculum probably has already occurred, but may need to be 
revisited as an alternate assessment is developed. 

Students must also have an opportunity to learn and be tested on the 
outcomes of schooling that are valued for ah students. Balance is critical 
here. Teachers in Kentucky report that statewide testing pushed them to 
focus on key intended outcomes (essential elements) of instruction. We do 
not want to return to the days of overly limited expectations for students 
with severe disabilities. An alternate assessment system should be designed 
to assess achievement toward pre -determined standards. Preferably, these 
would be the same standards measured by typical assessments. In any 
event, alternate assessments should measure high standards and target the 
goals for the student. 

Issue 9: What are the Cost Benefits? 

States incur considerable costs when they develop or revise their general 
assessment programs. Likewise, costs will be incurred when they develop 
alternate assessments for students. The cost per child can be very high 
given the low incidence of some disabilities. It is not expedient for states to 
construct separate tests for each kind of student with a disability, yet the 
very purpose of the alternate assessment is to develop more appropriate 
measures. The state must strike a balance among development cost, time, 
and responsiveness to idiosyncrasies. 

It can take considerable time to develop/design alternate assessments. It 
also takes time to administer the assessments. Efforts must be made to keep 
time commitments reasonably low. 

Issue 10: How Will Data be Collected? 

Assessors must decide how they will gather information on student 
achievement and development. Ysseldyke and Olsen (1996) have described 
many ways in which data may be collected, relying on the methodologies 
described earlier by Salvia and Ysseldyke (1995). Data may be gathered 



NCEO 



11 



Alternate 
assessments 
may require 
new training 
to bring 
school 
personnel 
up to speed 
with new 
assessment 
practices. 



through observation, recollection (interview, completion of checklists, self- 
report), testing, and record review. Assessors must make choices among 
these options, trying to balance cost, time, and reliability/validity. For 
example, use of multiple measures (e.g., performance’ event parent 
interview) will increase reliability, but will also increase cost. 

Issue 11: What Kind(s) of Data are to be Collected? 

Assessment is a process of collecting data for the purpose of making 
decisions about students and/or schools. Assessment personnel regularly 
debate the kinds of data to be collected. To the extent that standards are 
broad and can encompass all students, then it makes sense to measure the 
same content in the alternate assessment as is measured in typical 
assessments. It is necessary to modify how information is collected, but 
not what is collected. Thus, for example, data might be collected on 
mastery of science goals by means of an alternate portfolio rather than by 
means of performance events. 

Issue 12: How Do We Maintain Confidentiality? 

Alternate assessments typically are given to only a few students. While not 
unique to alternate assessment, confidentiality is an issue when the results 
of the alternate assessment are reported. Because there typically are only a 
few students in a district or region who take alternate assessments, their 
individual performances could be easily identifiable. The state will have to 
find ways of public reporting that make it difficult to track performance to 
individual students. 

Issue 13: How Do We Ensure that School Personnel are 

Trained to Administer, Score, Interpret and Use the 
Results of Alternate Tests? 

Alternate assessments involve assessment practices and new ways of 
collecting data for which school personnel have limited training. They 
require new training designed to bring people up to speed with the 
assessment practices, and training in curriculum and instructional 
approaches designed to achieve outcomes specified and measured by 
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alternate tests. Providing this training can be costly and time consuming, 
yet not providing training runs the risk of long term negative effects. 
Kentucky and Maryland have elected to involve their teachers of students 
with disabilities in development and scoring of the assessments. This 
involvement has had the benefit of merging training, development, and 
implementation in a cost-effective manner. 



Summary - 

National and most state legislation includes language specifying that 
standards, assessments, and accountability systems are for all students, 
including students with disabilities. States face significant challenges in 
their efforts to develop fully inclusive accountability systems. It is clear that 
alternate assessments will be needed for a small number (probably less than 
one percent) of students. States considering alternate assessments must take 
into account the 13 issues specified in this paper. The content, magnitude, 
and method of alternate assessment and the ways in which data are reported 
and used will have a significant effect on states’ development or 
maintenance of inclusive educational accountability systems. Failure to 
adequately address these issues could lead to standards for some, restricted 
policies, separate educational systems, divided accountability systems, and 
diminished educational outcomes for a large number of students. 
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